Skip to content

Add agent discoverability surfaces#18

Closed
mikedotexe wants to merge 3 commits into
mainfrom
codex/agent-discoverability-core
Closed

Add agent discoverability surfaces#18
mikedotexe wants to merge 3 commits into
mainfrom
codex/agent-discoverability-core

Conversation

@mikedotexe

@mikedotexe mikedotexe commented Apr 18, 2026

Copy link
Copy Markdown
Collaborator

What this PR adds

Static metadata and tooling that lets AI agents and crawlers self-bootstrap to FastNear's surfaces, plus a generator and audit pipeline to keep it consistent.

  • static/.well-known/agent-skills/{index.json + 4 SKILL.md} — agentskills.io v0.2.0 discovery directory (overview, surface-routing, auth, playbooks)
  • static/.well-known/api-catalog — RFC 8615 linkset for RPC, FastNear API, and NEARData (with service-desc, service-doc, status per service)
  • src/data/openapiSnapshots/{fastnear,neardata}.json (canonical, hand-vendored) → static/openapi/{fastnear,neardata}.json (build mirror via copyOpenApiSnapshots() in scripts/generate-ai-surfaces.js)
  • static/_headers — advertises the discovery surfaces via HTTP Link headers (RFC 8615 / agent-skills) on every route
  • static/_worker.js — serves .md mirrors via content negotiation (Accept: text/markdown) and adds the matching Link headers
  • static/robots.txt — adds Cloudflare AI Content Signals: Content-Signal: search=yes, ai-input=yes, ai-train=yes
  • scripts/generate-ai-surfaces.js — extended (~233 lines) to drive the artifacts above; SKILL.md files also pick up the existing "About FastNear" footer CTA
  • scripts/audit-indexing-surface.js — extended (~509 lines) to validate the new static surface area; runs in ci:locale-quality post-build (no new CI gating beyond what's already there)
  • scripts/audit-agent-readiness-live.js + audit:agent-readiness:live — manual live smoke test against the deployed origin, validates the Link headers and .md content negotiation are wired correctly

Hosting

docs.fastnear.com is on Cloudflare Pages (visible from the repo's Cloudflare Pages GitHub status check), so static/_headers and static/_worker.js are load-bearing — they execute on the edge once this merges. No infra change required.

Current branch status

Rebased onto current main (post #20 and #21 example-tightening merges). Two real fixes surfaced from the rebase + regen:

  1. The four SKILL.md files now correctly include the "About FastNear" footer CTA that landed on main after this branch was first opened.
  2. static/ru/** markdown caught up to main's tightened example content.

Verification

  • yarn build clean (en + ru both compile)
  • node scripts/audit-indexing-surface.js clean against local build (Indexing surface audit passed for 105 explicit docs routes, 78 docs operation pages, and 78 hosted operation pages.)
  • yarn generate:ai-surfaces is idempotent: re-running produces no further drift
  • OpenAPI build mirror is in sync with the canonical source (formatting differs — escaping vs raw em-dash from JSON.stringify re-encoding — but semantically identical)
  • Live audit script is manual-only: not present in ci:locale-quality or any other ci:*/audit:* aggregate
  • Spot-checked: SKILL.md files end with the "About FastNear" footer block; robots.txt has the Content-Signal line

Test plan

  • Rebase onto current main is clean
  • yarn build exits 0 for both locales
  • node scripts/audit-indexing-surface.js exits 0 against the local build tree
  • Generator output is idempotent (second run produces no diff)
  • Cloudflare Pages preview deploy succeeds
  • Post-merge: SITE_ORIGIN=https://docs.fastnear.com node scripts/audit-agent-readiness-live.js exits clean (validates Link headers, .well-known/api-catalog, .well-known/agent-skills/index.json, and .md content negotiation are live)

Out of scope for this PR

  • Wiring audit:agent-readiness:live into CI (kept manual until we trust the live signal across a few deploys)
  • Auto-pulling upstream OpenAPI specs into src/data/openapiSnapshots/ (manual snapshot maintenance for now; document a sync cadence in a follow-up)

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Apr 18, 2026

Copy link
Copy Markdown

Deploying builder-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0fef871
Status: ✅  Deploy successful!
Preview URL: https://7018604e.builder-docs.pages.dev
Branch Preview URL: https://codex-agent-discoverability.builder-docs.pages.dev

View logs

mikedotexe and others added 3 commits April 25, 2026 11:55
This live smoke test asserts HTTP Link headers and content-negotiated .md
serving advertised via static/_headers and static/_worker.js, which only
execute on Cloudflare Pages with Workers/Pages Functions. Note this in a
header comment so future readers know that a failure against the GitHub
Pages origin is by design and not a regression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mikedotexe

Copy link
Copy Markdown
Collaborator Author

Closing to clear the board — the branch codex/agent-discoverability-core is preserved, so no work is lost and this can be revived as scoped work when agent-discoverability is actually on the roadmap.

Why not merge as-is:

  • Stalled ~3 months and 25 commits behind main. The companion rebase/agent-discoverability-core branch turned out to be an identical duplicate of this PR's head, so that rebase was started but never completed.
  • Large surface (+8,488/-963 across 39 files) that rewrites the CI gate (scripts/audit-indexing-surface.js +503) and adds a Cloudflare edge static/_worker.js + static/_headers. That's infra worth a deliberate review rather than a bulk merge — particularly right after a Cloudflare Pages deployment incident on this repo.
  • The vendored src/data/openapiSnapshots/ predate docs(rpc): nearcore 2.13.0 RPC reference + technical descriptions #28 (nearcore 2.13.0), so they would need regenerating anyway.

Note the human-facing agent surfaces already landed on main (AGENTS.md, docs/agents/* + RU twins, scripts/generate-ai-surfaces.js). What's still un-landed here is the machine-discoverable layer (.well-known/agent-skills/, api-catalog, edge worker) — a good standalone follow-up.

@mikedotexe mikedotexe closed this Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant